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DETAILED ACTION 
Claim Rejections - 35 USC § 112 
The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 14-17, 25-26 and 32-33 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. The claim 13 recites "a digital data storage medium on which 
is stored a program for implementing the method of claim 1. The claim 14 is directed to neither a 
"process" nor a "manufacture." Clarification is required. 

Claim 14 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. The claim 14 recites "a computer program making a computer function as an 
image processing apparatus of claim 1 " The claim language is vague and ambiguous as it cannot 
be ascertained whether the claim 14 is an apparatus claim or a process claim. Clarification is 
required. 

The claims 15-17, 25-26 and 32-33 are subject to the same rationale of rejection set forth 
in the claim 14. 

Claim 20 recites the claim limitation that a pixel size of one side of the virtual space 
image generated by said generation unit assumes an arbitrary value which has, as an upper limit, 
a product of a length of one side and a resolution of an image that the print unit can output. Since 
a length of one side of the virtual space image may be equal to a resolution value of an image 
that the print unit can output. A pixel size is much less than a length of one side of the virtual 
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space or a resolution of an image that the print unit can output. It does not make sense by reciting 
the limitation of "a product" as an upper limit. Clarification is required. 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claim 14-17, 25-26and 32-33 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

The claim 14 recites "a computer program making a computer function as an image 
processing apparatus of claim 1 The claim 14 is directed to neither a "process" nor a 
"manufacture," but rather embrace and overlap two different statutory classes of invention set 
forth in 35 U.S.C. 101, which is drafted so as to set requirements for the statutory classes of 
invention in the alternative only. See, Ex Parte Lyell, 17 USPQ2d 1548 (Bd. Pat. App. & Inter. 
1990). See also, MPEP 2173.05(p). 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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Claims 1-33 are rejected under 35 U.S.C. 103(a) as being unpatentable over Steven 
Feiner, Blair Maclntre, Tobias Hollerer, and Anthony Webster, "Prototyping 3D Mobile 
Augmented Reality Systems for Exploring the Urban Environment", Personal Technologies, vol. 
1, no. 4, pp. 208-217, 1997 (hereinafter Feiner) in view of Filo et al. U.S. Patent No. 6,215,498 
(hereinafter Filor). 

Re Claims 1, 12, 18-19 and 27: 

Feiner teaches an image processing apparatus which is connected to print unit adapted to 
print an image, and can output an image to be printed to the print unit (See Page 75 at Section 3, 
wherein Feiner discloses a browser running on the handheld computer that generates a page of 
mixed reality on the fly and the web browser allows the page be printed on a printer connected 
to the handheld computer on a network of base stations on campus wherein the network 
inherently has a plurality of printers; see column 2 of Page 78), comprising: 

Image sensing unit adapted to capture an image of a real space (Page 78, Section 4.1 
including a position and orientation tracker with HMD); 

Generation unit adapted to generate an image of a virtual space in accordance with a 
position and orientation of said image sensing unit (Page 78, Section 4.1 including a position 
and orientation tracker with HMD and the graphics image of a virtual space is generated in 
accordance with the position and orientation of the trackers), which serves as a viewpoint of the 
virtual space, on a coordinate system shared by the real space and the virtual space (See Section 
42, based on the user input and a database of information about campus buildings, it generates 
the graphics that are overlaid on the real world by the head-worn display. See Section 43 
wherein the images are generated by the CCD camera. Overlaying virtual objects or graphics 
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and textual texts on the real world images shot by the sensing unit on the head-worn display; see 
Section 3 wherein the real world image were shot through the head-worn display); 

Storage unit adapted to superpose and storing the image of the real space captured by said 
image sensing unit and the image of the virtual space generated by said generation unit to store a 
mixed reality space image formed by superposing the two images (See Section 4.2, based on the 
user input and a database of information about campus buildings, it generates the graphics that 
are overlaid on the real world by the head-worn display. See Section 4.3 wherein the images are 
generated by the CCD camera. See Section 3; for example, overlaying the textual labels on 
automatically recognized campus buildings shown in Figures 2 and 3 wherein the building is 
selected based on the viewpoint of the user y s head position and orientation and the mixed reality 
image is stored at least in a cache memory or the frame buffer of the head-worn display and 
displayed on the head-worn display. As shown in Fig. 7, the HTTP server has two components, 
the campus information server, responsible for the dynamic generation of HTML pages, and a 
caching proxy server. The purpose of the proxy server is to cache the data returned by external 
HTTP requests to mitigate the slowness of the radio network link. Commonly accessed pages, 
such as department home pages and building descriptions, can be pre-cached without relying on 
the browser 's own caching mechanisms); 

Instruction unit adapted to instruct whether or not the mixed reality space image stored in 
said storage unit is to be printed (At Page 75 and Section 3, Feiner discloses a browser running 
on the handheld computer that generates a page of mixed reality on the fly and the web browser 
allows the page be printed on a printer connected to the handheld computer on a network of 
base stations on campus wherein the network inherently has a plurality of printers; see column 2 
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of Page 78; On the web page, the user instructs whether or not the mixed reality space image is 
to be printed or not and the user interface or the web browser allowing the instruction to be 
made constitutes the instruction unit allowing the user to instruct whether or not the mixed 
reality space image is to be printed or not Since the superposed image is displayed on a web 
browser, it can be printed on a printer on campus network); and 

Output unit adapted to output the mixed reality space image stored in said storage unit to 
the print unit upon reception of a print instruction of the mixed reality image from said 
instruction unit (At Page 75 and Section 3, wherein Feiner discloses a browser running on the 
handheld computer that generates a page of mixed reality on the fly and the web browser allows 
the page be printed on a printer connected to the handheld computer on a network of base 
stations on campus wherein the network inherently has a plurality of printers; see column 2 of 
Page 78; Since the superposed image is displayed on a web browser, it can be printed on a 
printer on campus network). 

Although Feiner does not explicitly disclose "the print unit", Feiner' s apparatus is 
connected to the campus network and the campus network inherently has a plurality of printers 
connected to allowing Feiner' s web browser to print the mixed reality image upon user's 
instruction. Nevertheless, Filo discloses the print unit (See Filo column 11, lines 10-26). Filo 
teaches other claim limitations set forth in the claim 1 and similar claims as well by disclosing 
the superposition of the virtual objects with the real objects. 

It would have been obvious to one of the ordinary skill in the art to have used a printer to 
print the mixed reality image for review at a later time using the web browser and a network 
printer (See Feiner Page 78 and Filo column 11, lines 10-26). One of the ordinary skill in the art 
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knows how to connect a printer to a handheld computer. One of the ordinary skill in the art 
would have been motivated to do so to print the mixed reality image, e.g., in the interest of 
showing the campus building to a third party (See Feiner Page 78 and Filo column 11, lines 10- 
26). 

With respect to the claims 18-19 and 27, Feiner teaches in Page 79 that the HTTP server 
can call a network object method that selects corresponding graphical items on the head-worn 
display. Thus data selection works in both directions: from the backpack PC to the handheld PC 
by launching relevant URLs from the head-worn display's menus and vice versa by selecting 
buildings, departments, etc on the head-worn display from a link on the handheld's browser. 
Feiner teaches that the tour application continuously receives input from the GPS position 
tracker and the orientation tracker and therefore, when the instruction is received from said 
instruction unit (e.g., in a graphical user interface), the mixed reality image is re-generated using 
a second calculation method and sequence such as the blending the graphics image so generated 
with a different building image and different labels are generated to overlay on the building 
image and thus requiring different calculation methods for calculating the network objects or 
generating the graphics images to be overlaid on the building images. Figs. 2-3 show the mixed 
reality images according to a first viewpoint and a second viewpoint. Based on this input and a 
database of information about campus buildings, it generates the graphics that are overlaid on the 
real world by the head-worn display. 

Claim 2: 

Feiner teaches a management unit adapted to manage state information of the virtual 
space, and in that said management unit updates the state information of the virtual space in 
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accordance with input information or at each predetermined timing, and said generation unit 
generates a virtual space image according to the updated state information of the virtual space 
(e.g., Section 4.2 wherein a system that provides language-level support for distributed virtual 
environment and CORTERIE is based on the distributed data-object paradigm for distributed 
shared memory and the tour application running on the backpack PC is response for generating 
the graphics and presenting it on the head-worn display and the HTTP server can call a network 
object method that selects corresponding graphical items on the head-worn display and the 
handheld display controls the head-worn display. The tour application continuously receives 
input from the GPS position tracker and the orientation tracker and thus continuously updating 
the mixed reality image composed by the graphics image generated from the handheld computer 
and the HTTP server and the building image shot from the camera; see Figs. 2-3. It also takes 
user input from the trackpad that is physically attached to the back of the handheld PC. Based on 
this input and a database of information about campus buildings, it generates the graphics that 
are overlaid on the real world by the head-worn display. See also Section 3, when a building is 
selected, a conical green compass pointer appears at the bottom of the head-worn display, 
oriented in the building's direction. The pointer turns red if the building is more than 90 degrees 
away from the user 's head orientation. This allows the user to find the building more easily if 
they turn away from it. The pointer is especially useful for finding buildings selected from the 
hand-held computer. This is made possible by our custom HTTP server, which can tell the 
backpack computer to select a building on the head-worn display. See Section 4.2, the replicated 
shared objects support asynchronous data propagation with atomic serializable updates and 
asynchronous notification of updates). 
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Claim 3: 

Feiner further discloses that said management unit is an independent apparatus connected 
to said image processing apparatus via a network, and the state information of the virtual space 
updated by said management unit can be received by image processing apparatuses which are 
equivalent to said image processing apparatus, and are connected to the network (e.g., Section 
4.2 wherein a system that provides language-level support for distributed virtual environment 
and CORTERIE is based on the distributed data-object paradigm for distributed shared memory 
and the tour application running on the backpack PC is response for generating the graphics 
and presenting it on the head-worn display and the HTTP server can call a network object 
method that selects corresponding graphical items on the head-worn display and the handheld 
display controls the head-worn display. The tour application continuously receives input from 
the GPS position tracker and the orientation tracker. It also takes user input from the trackpad 
that is physically attached to the back of the handheld PC. Based on this input and a database of 
information about campus buildings, it generates the graphics that are overlaid on the real 
world by the head-worn display). 

Claim 4: 

Feiner further discloses that display unit adapted to display the mixed reality space image 
stored in said storage unit (e.g., Section 4.2 wherein a system that provides language-level 
support for distributed virtual environment and CORTERIE is based on the distributed data- 
object paradigm for distributed shared memory and the tour application running on the 
backpack PC is response for generating the graphics and presenting it on the head-worn display 
and the HTTP server can call a network object method that selects corresponding graphical 
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items on the head-worn display and the handheld display controls the head-worn display. The 
tour application continuously receives input from the GPS position tracker and the orientation 
tracker. It also takes user input from the trackpad that is physically attached to the back of the 
handheld PC. Based on this input and a database of information about campus buildings, it 
generates the graphics that are overlaid on the real world by the head-worn display). 
Claim 5: 

Feiner further discloses that said image sensing unit is an HMD which comprises a 
display unit for displaying the mixed reality space image stored in said storage unit, 
said apparatus further comprises a sensor for measuring a position and orientation near a 
viewpoint position of an observer who wears the HMD on the coordinate system, and said 
generation unit calculates based on the position and orientation near the viewpoint position on 
the coordinate system as measurement results of said sensor, and a constant which is measured in 
advance (e.g., Section 4.2 wherein a system that provides language-level support for distributed 
virtual environment and CORTERIE is based on the distributed data-object paradigm for 
distributed shared memory and the tour application running on the backpack PC is response for 
generating the graphics and presenting it on the head-worn display and the HTTP server can 
call a network object method that selects corresponding graphical items on the head-worn 
display and the handheld display controls the head-worn display. The tour application 
continuously receives input from the GPS position tracker and the orientation tracker. It also 
takes user input from the trackpad that is physically attached to the back of the handheld PC. 
Based on this input and a database of information about campus buildings, it generates the 
graphics that are overlaid on the real world by the head-worn display). 
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Claim 6: 

Feiner further discloses that the viewpoint includes a first viewpoint and a second 
viewpoint different from the first viewpoint, and said generation unit generates an image of the 
virtual space according to a position and orientation of the first viewpoint on the coordinate 
system using the position and orientation near the viewpoint position on the coordinate system as 
the position and orientation of the first viewpoint, and generates an image of the virtual space 
according to a position and orientation of the second viewpoint on the coordinate system using a 
position which is shifted a predetermined distance from the position of the first viewpoint and 
the orientation of the first viewpoint as the position and orientation of the second viewpoint (e.g., 
Section 42 wherein a system that provides language-level support for distributed virtual 
environment and CORTERIE is based on the distributed data-object paradigm for distributed 
shared memory and the tour application running on the backpack PC is response for generating 
the graphics and presenting it on the head-worn display and the HTTP server can call a network 
object method that selects corresponding graphical items on the head-worn display and the 
handheld display controls the head-worn display. The tour application continuously receives 
input from the GPS position tracker and the orientation tracker. It also takes user input from the 
trackpad that is physically attached to the back of the handheld PC and Figs. 2-3 show the mixed 
reality images according to a first viewpoint and a second viewpoint Based on this input and a 
database of information about campus buildings, it generates the graphics that are overlaid on 
the real world by the head-worn display). 

Claim 7: 
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Feiner further discloses that said image sensing unit is an image sensing device, which 
comprises a display unit for displaying the mixed reality space image stored in said storage unit, 
and is attached to a fixed post to be rotatable in a desired direction, said apparatus further 
comprises a sensor for measuring an orientation of the attached portion of the image sensing 
device on the coordinate system, and said generation unit uses a measurement result of said 
sensor as a orientation of the viewpoint (e.g., Section 4.2 wherein a system that provides 
language-level support for distributed virtual environment and CORTERIE is based on the 
distributed data-object paradigm for distributed shared memory and the tour application 
running on the backpack PC is response for generating the graphics and presenting it on the 
head-worn display and the HTTP server can call a network object method that selects 
corresponding graphical items on the head-worn display and the handheld display controls the 
head-worn display. The tour application continuously receives input from the GPS position 
tracker and the orientation tracker. It also takes user input from the trackpad that is physically 
attached to. the back of the handheld PC. Based on this input and a database of information 
about campus buildings, it generates the graphics that are overlaid on the real world by the 
head-worn display). 

Re Claims 8 and 13: 

Feiner teaches an image processing apparatus, which is connected to a plurality of 
apparatuses each of which generates a mixed reality space image formed by superimposing 
images of a virtual space and real space (e.g., Section 4.2 wherein a system that provides 
language-level support for distributed virtual environment and CORTERIE is based on the 
distributed data-object paradigm for distributed shared memory and the tour application 
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running on the backpack PC is response for generating the graphics and presenting it on the 
head-worn display and the HTTP server can call a network object method that selects 
corresponding graphical items on the head-worn display and the handheld display controls the 
head-worn display wherein calling a network object requires a selection of one of the plurality of 
apparatuses in the network to obtain a network object resident in an apparatus in the network. 
The tour application continuously receives input from the GPS position tracker and the 
orientation tracker. It also takes user input from the trackpad that is physically attached to the 
back of the handheld PC. Based on this input and a database of information about campus 
buildings, it generates the graphics that are overlaid on the real world by the head-worn 
display), is connected to print unit adapted to print an image, and can output an image to be 
printed to the print unit {See Page 75 at Section 3, wherein Feiner discloses a browser running 
on the handheld computer that generates a page of mixed reality on the fly and the web browser 
allows the page be printed on a printer connected to the handheld computer on a network of 
base stations on campus wherein the network inherently has a plurality of printers; see column 2 
of Page 78), comprising: 

Selection unit adapted to select a desired one of the plurality of apparatuses (e.g., Section 
4.2 wherein a system that provides language-level support for distributed virtual environment 
and CORTERIE is based on the distributed data-object paradigm for distributed shared memory 
and the tour application running on the backpack PC is response for generating the graphics 
and presenting it on the head-worn display and the HTTP server can call a network ob ject 
method that selects corresponding graphical items on the head-worn display and the handheld 
display controls the head-worn display wherein calling a network object requires a selection of 
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one of the plurality of apparatuses in the network to obtain a network object resident in an 
apparatus in the network. The tour application continuously receives input from the GPS 
position tracker and the orientation tracker. It also takes user input from the trackpad that is 
physically attached to the back of the handheld PC. Based on this input and a database of 
information about campus buildings, it generates the graphics that are overlaid on the real 
world by the head-worn display); 

Instruction unit adapted to instruct whether or not the mixed reality space image obtained 
from the apparatus selected by said selection unit is to be printed (At Page 75 and Section 3, 
Feiner discloses a browser running on the handheld computer that generates a page of mixed 
reality on the fly and the web browser allows the page be printed on a printer connected to the 
handheld computer on a network of base stations on campus wherein the network inherently has 
a plurality of printers; see column 2 of Page 78; On the web page, the user instructs whether or 
not the mixed reality space image is to be printed or not and the user interface or the web 
browser allowing the instruction to be made constitutes the instruction unit allowing the user to 
instruct whether or not the mixed reality space image is to be printed or not. Since the 
superposed image is displayed on a web browser, it can be printed on a printer on campus 
network); and 

Output unit adapted to output the mixed reality space image to the print unit upon 
reception of a print instruction of the mixed reality image from said instruction unit (At Page 75 
and Section 3, wherein Feiner discloses a browser running on the handheld computer that 
generates a page of mixed reality on the fly and the web browser allows the page be printed on a 
printer connected to the handheld computer on a network of base stations on campus wherein 
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the network inherently has a plurality of printers; see column 2 of Page 78; Since the superposed 
image is displayed on a web browser, it can be printed on a printer on campus network). 

Although Feiner does not explicitly disclose "the print unit", Feiner' s apparatus is 
connected to the campus network and the campus network inherently has a plurality of printers 
connected to allowing Feiner' s web browser to print the mixed reality image upon user's 
instruction. Nevertheless, Filo discloses the print unit (See Filo column 11, lines 10-26). Filo 
teaches other claim limitations set forth in the claim 1 and similar claims as well by disclosing 
the superposition of the virtual objects with the real objects. 

It would have been obvious to one of the ordinary skill in the art to have used a printer to 
print the mixed reality image for review at a later time using the web browser and a network 
printer (See Feiner Page 78 and Filo column 11, lines 10-26). One of the ordinary skill in the art 
knows how to connect a printer to a handheld computer. One of the ordinary skill in the art 
would have been motivated to do so to print the mixed reality image, e.g., in the interest of 
showing the campus building to a third party (See Feiner Page 78 and Filo column 11, lines 10- 
26). 

Claim 9: 

Feiner teaches that generation unit adapted to generate one image which includes the 
mixed reality space image obtained from the apparatus selected by said selection unit, and in that 
said output unit outputs the one image to the print unit upon reception of the print instruction 
from said instruction unit (e.g., Section 4.2 wherein a system that provides language4evel 
support for distributed virtual environment and CORTERIE is based on the distributed data- 
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object paradigm for distributed shared memory and the tour application running on the 
backpack PC is response for generating the graphics and presenting it on the head-worn display 
and the HTTP server can call a network object method that selects corresponding graphical 
items on the head-worn display and the handheld display controls the head-worn display 
wherein calling a network object requires a selection of one of the plurality of apparatuses in the 
network to obtain a network object resident in an apparatus in the network The tour application 
continuously receives input from the GPS position tracker and the orientation tracker. It also 
takes user input from the trackpad that is physically attached to the back of the handheld PC. 
Based on this input and a database of information about campus buildings, it generates the 
graphics that are overlaid on the real world by the head-worn display, e.g., Section 4.2 wherein 
a system that provides language-level support for distributed virtual environment and 
CORTERIE is based on the distributed data-object paradigm for distributed shared memory and 
the tour application running on the backpack PC is response for generating the graphics and 
presenting it on the head-worn display and the HTTP server can call a network object method 
that selects corresponding graphical items on the head-worn display and the handheld display 
controls the head-worn display wherein calling a network object requires a selection of one of 
the plurality of apparatuses in the network to obtain a network object resident in an apparatus in 
the network. The tour application continuously receives input from the GPS position tracker and 
the orientation tracker. It also takes user input from the trackpad that is physically attached to 
the back of the handheld PC. Based on this input and a database of information about campus 
buildings, it generates the graphics that are overlaid on the real world by the head-worn 
display). 
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Claim 10: 

Feiner teaches that, when a plurality of mixed reality space images are obtained from the 
apparatus selected by said selection unit, said generation unit reduces the mixed reality space 
images and generates one image which includes the reduced mixed reality space images (e.g., 
Section 4.2 wherein a system that provides language-level support for distributed virtual 
environment and CORTERIE is based on the distributed data-object paradigm for distributed 
shared memory and the tour application running on the backpack PC is response for generating 
the graphics and presenting it on the head-worn display and the HTTP server can call a network 
object method that selects corresponding graphical items on the head-worn display and thus 
selecting the mixed reality space image wherein the handheld display controls the head-worn 
display wherein calling a network object requires a selection of one of the plurality of 
apparatuses in the network to obtain a network object resident in an apparatus in the network 
The tour application continuously receives input from the GPS position tracker and the 
orientation tracker. It also takes user input from the trackpad that is physically attached to the 
back of the handheld PC, Based on this input and a database of information about campus 
buildings, it generates the graphics that are overlaid on the real world by the head-worn display, 
e.g., Section 4.2 wherein a system that provides language-level support for distributed virtual 
environment and CORTERIE is based on the distributed data-object paradigm for distributed 
shared memory and the tour application running on the backpack PC is response for generating 
the graphics and presenting it on the head-worn display and the HTTP server can call a network 
object method that selects corresponding graphical items on the head-worn display and the 
handheld display controls the head-worn display wherein calling a network object requires a 
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selection of one of the plurality of apparatuses in the network to obtain a network object resident 
in an apparatus in the network. The tour application continuously receives input from the GPS 
position tracker and the orientation tracker. It also takes user input from the trackpad that is 
physically attached to the back of the handheld PC. Based on this input and a database of 
information about campus buildings, it generates the graphics that are overlaid on the real 
world by the head-worn display). 
Claim 11: 

Feiner further discloses that display unit adapted to display the one image generated by 
said generation unit (e.g.. Section 4.2 wherein a system that provides language-level support for 
distributed virtual environment and CORTERIE is based on the distributed data-object paradigm 
for distributed shared memory and the tour application running on the backpack PC is response 
for generating the graphics and presenting it on the head-worn display and the HTTP server can 
call a network object method that selects corresponding graphical items on the head-worn 
display and the handheld display controls the head-worn display. The tour application 
continuously receives input from the GPS position tracker and the orientation tracker. It also 
takes user input from the trackpad that is physically attached to the back of the handheld PC. 
Based on this input and a database of information about campus buildings, it generates the 
graphics that are overlaid on the real world by the head-worn display). 

Claim 14: 

The claim 14 is subject to the same rationale of rejection set forth in the claim 1. 
Claim 15: 

The claim 15 is subject to the same rationale of rejection set forth in the claim 12. 
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Claim 16: 

The claim 16 is subject to the same rationale of rejection set forth in the claim 1. 
Claim 17: 

The claim 17 is subject to the same rationale of rejection set forth in the claim 12. 
Claim 20: 

Feiner teaches that a pixel size of one side of the virtual space image generated by said 
generation unit assumes an arbitrary value which has, as an upper limit, a product of a length of 
one side and a resolution of an image that the print unit can output (Feiner Section 4.1). 

Re Claims 21 and 28: 

Feiner teaches that management unit adapted to manage state information of the real and 
virtual spaces, and in that said management unit updates the state information of the virtual space 
in accordance with input information or at each predetermined timing, and said generation unit 
successively generates virtual space images in accordance with the updated state information of 
the real and virtual spaces (e.g., Section 4.2 wherein a system that provides language-level 
support for distributed virtual environment and CORTERIE is based on the distributed data- 
object paradigm for distributed shared memory and the tour application running on the 
backpack PC is response for generating the graphics and presenting it on the head-worn display 
and the HTTP server can call a network object method that selects corresponding graphical 
items on the head-worn display and the handheld display controls the head-worn display. The 
tour application continuously receives input from the GPS position tracker and the orientation 
tracker. It also takes user input from the trackpad that is physically attached to the back of the 
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handheld PC. Based on this input and a database of information about campus buildings, it 
generates the graphics that are overlaid on the real world by the head-worn display). 
Re Claims 22 and 29: 

Feiner further teaches that said management unit is an independent apparatus connected 
to said image processing apparatus via a network, and the state information of the virtual space 
updated by said management unit can be received by image processing apparatuses which are 
equivalent to said image processing apparatus, and are connected to the network (e.g., Section 
4.2 wherein a system that provides language-level support for distributed virtual environment 
and CORTERIE is based on the distributed data-object paradigm for distributed shared memory 
and the tour application running on the backpack PC is response for generating the graphics 
and presenting it on the head-worn display and the HTTP server can call a network object 
method that selects corresponding graphical items on the head-worn display and the handheld 
display controls the head-worn display. The tour application continuously receives input from 
the GPS position tracker and the orientation tracker. It also takes user input from the trackpad 
that is physically attached to the back of the handheld PC. Based on this input and a database of 
information about campus buildings, it generates the graphics that are overlaid on the real 
world by the head-worn display). 

Re Claims 23 and 30: 

Feiner further teaches that said apparatus can be simultaneously used by at least one user, 
and has said image sensing unit, said generation unit, said composition unit, said display unit, 
and detection unit adapted to detect a viewpoint position and visual axis direction of a user, 
which are prepared for each user (e.g., Section 4.2 wherein a system that provides language-level 
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support for distributed virtual environment and CORTERIE is based on the distributed data- 
object paradigm for distributed shared memory and the tour application running on the 
backpack PC is response for generating the graphics and presenting it on the head-worn display 
and the HTTP server can call a network object method that selects corresponding graphical 
items on the head-worn display and the handheld display controls the head-worn display. The 
tour application continuously receives input from the GPS position tracker and the orientation 
tracker. It also takes user input from the trackpad that is physically attached to the back of the 
handheld PC. Based on this input and a database of information about campus buildings, it 
generates the graphics that are overlaid on the real world by the head-worn display). 
Re Claims 24 and 31: 

Feiner further teaches that said display unit and said image sensing unit, forms or form a 
head-mounted display device to be mounted on a head of the user (Fig. 1). 
Claims 25-26: 

The claims 25-26 are subject to the same rationale of rejection set forth in the claim 18. 
Claims 32-33: 

The claims 32-33 are subject to the same rationale of rejection set forth in the claim 27. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (571) 272-7665. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on (571) 272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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